Add magic bytes validation for Mach-O binaries in DotnetHostHelper #15230
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: nohwnd <[email protected]>
Copilot
AI
changed the title
Copilot finished work on behalf of
nohwnd
July 17, 2025 09:06
nohwnd
approved these changes
Jul 17, 2025
Youssef1313
approved these changes
Jul 17, 2025
fhnaseer
approved these changes
Jul 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR ports the magic bytes validation fix for Mach-O binaries from TestFX PR #6033 to address the issue where
GetMuxerArchitectureByMachoOnMacwas reading magic bytes but never validating them against known Mach-O magic numbers.Problem
The
GetMuxerArchitectureByMachoOnMacmethod inDotnetHostHelper.cswas reading magic bytes from binary files but never validating them against known Mach-O magic numbers. This meant the method could attempt to process non-Mach-O binaries as if they were valid Mach-O files, potentially leading to incorrect architecture detection.Solution
Added validation of magic bytes to ensure we're actually looking at a valid Mach-O binary before proceeding with CPU type detection.
Changes Made
Added Mach-O magic number constants based on the Wikipedia Mach-O specification:
MachOMagic32BigEndian = 0xfeedface(32-bit big-endian)MachOMagic64BigEndian = 0xfeedfacf(64-bit big-endian)MachOMagic32LittleEndian = 0xcefaedfe(32-bit little-endian)MachOMagic64LittleEndian = 0xcffaedfe(64-bit little-endian)MachOMagicFatBigEndian = 0xcafebabe(multi-architecture big-endian)Added validation logic that checks the magic bytes against all valid Mach-O magic numbers before attempting architecture detection
Enhanced logging to report invalid magic bytes for debugging purposes
Implementation Details
The implementation aligns with the existing pattern in the codebase where CPU types also have both "Magic" and "Cigam" (byte-swapped) variants to handle different endianness.
Fixes #15226.
Warning
Firewall rules blocked me from connecting to one or more addresses
I tried to connect to the following addresses, but was blocked by firewall rules:
yttvsblobprodcus357.vsblob.vsassets.io/home/REDACTED/work/vstest/vstest/.dotnet/dotnet msbuild /m /nologo /clp:Summary /v:minimal /nr:true /warnaserror /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=false /home/REDACTED/work/vstest/vstest/artifacts/toolset/restore.proj /t:__WriteToolsetLocation /clp:ErrorsOnly;NoSummary /p:__ToolsetLocationOutputFile=/home/REDACTED/work/vstest/vstest/artifacts/toolset/10.0.0-beta.25358.3.txt(dns block)If you need me to access, download, or install something from one of these locations, you can either:
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.